1. Technical Field
The invention is related to a system for mosaicing images, and in particular, to a system and method for minimizing object distortions and ghosting caused by image parallax.
2. Related Art
In general, image mosaics are a combination of two or more overlapping images that serve to present an overall view of a scene from perspectives other than those of the individual images used to generate the mosaic. In other words, image-based rendering techniques such as the creation of image mosaics are used to render photorealistic novel views from collections of real or pre-rendered images which allow a user or viewer to look in any desired direction. Such novel views are useful for virtual travel, architectural walkthroughs, video games, or simply for examining a scene or area from perspectives not originally captured or otherwise rendered. Typically, better final mosaicing results for a given scene or area are achieved by using many overlapping images having a large percentage of overlap between the images.
Unfortunately, using large sets of overlapping images having a high degree of overlap for generating mosaics is typically computationally expensive. Further, where the set of overlapping images available for generating a mosaic comprises a sparse or limited set of images taken at slightly displaced locations, the problem of ghosting due to the presence of parallax becomes a major concern. In general, ghosting can be described as a visual artifact resulting from parallax that is frequently observed when images captured from different camera positions are either stitched, mosaiced, or otherwise combined. Specifically, any deviations from a pure parallax-free motion model or an ideal pinhole camera model can result in local misregistrations between the combined images. These misregistrations are typically visible as a loss of detail, such as blurring, or as two or more overlapping semi-transparent regions in the mosaiced images, i.e., ghosting.
There are several existing schemes for addressing ghosting when mosaicing images. For example, one conventional scheme uses a local patch-based deghosting technique in an attempt to address the problem. This scheme provides a system for constructing panoramic image mosaics from sequences of images. This scheme constructs a full view panorama using a rotational mosaic representation that associates a rotation matrix and, optionally, a focal length, with each input image in a sequence of images.
This scheme then uses a patch-based alignment algorithm which uses motion models to align two sequential images. In order to reduce accumulated registration errors between such images, a global alignment, or “block adjustment” is first applied to the whole sequence of images, which results in an optimally registered image mosaic. To compensate for small amounts of motion parallax introduced by translations of the camera and other unmodeled distortions, a local alignment technique for deghosting the combined images is used. This local alignment technique warps each image based on the results of pairwise local image registrations. Combining both the global and local alignment, serves to improve the quality of image mosaics generated using this scheme.
Unfortunately, while useful, because the aforementioned patch-based deghosting technique is purely image-based, it is only capable of addressing small amounts of motion parallax. Consequently, this scheme can not fully address significant parallax problems. Further, the corrective warping used in this patch-based deghosting technique often produces unrealistic-looking results. In addition, the patch-based deghosting technique summarized above tends to be computationally expensive.
Another conventional scheme for addressing the problem of parallax induced ghosting in stitched or mosaiced images involves the use of dense sampling to overcome the ghosting problem. Effectively, this dense sampling requires the use of images having significant overlapping regions. Specifically, this scheme provides for synthesizing an image from a new viewpoint using data from multiple overlapping reference images. This synthesized image is constructed from a dataset which is essentially a single image that is produced by combining samples from multiple viewpoints into a single image. Unfortunately, this scheme can not provide a satisfactory solution in the case of sparse sampling, such as where overlap between images is 50% or less and where parallax is a significant concern. In addition, because of the dense sampling, the aforementioned scheme tends to be computationally expensive.
Therefore, what is needed is a computationally efficient system and method for deghosting image mosaics. Further, this system and method should be capable of deghosting image mosaics even in the case where there is significant parallax, or where there is limited overlap between images used for creating the image mosaics.